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ABSTRACT 

This paper describes a computerized system for the evaluation of 
achievement in Title I programs. Developed originally for use in Iowa, 
it is presently being used in some form in four states. Local districts 
report raw scores to the State Department of Education, along with proper 
test identification. The system enters raw scores and other identifying 
information and produces summary reports by school building and district. 
It also gives a diagnosis of selected evaluation procedures and what 
percent of students were affected if improper procedures were used. 

The system has both reduced reporting burden for local districts and 
increased the accuracy of the aggregated state reports. 
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This paper describes a system designed and developed for the purpose 
of simplifying and improving scoring and summarizing the achievement of 
students participating in Title I programs. The system which we describe 
is presently in use in some form in four states, and is being considered 
for use in 6 others. It was developed for use in the state of Iowa, and 
has been in use in that state for almost 4 years. 

Background on the Title I Evaluation and Reporting System 

Title I programs are established with federal funds allocated for 
special or additional training beyond the regular classroom primarily in 
the basic skills of reading, mathematics, and language arts. Students are 
selected for Title I programs based on factors that include family income, 
scores on tests, and teacher recommendations* The impact of these prog- 
rams must, by federal mandate, be evaluated and the result of this evalua- 
tion reported by the state to federal agencies. 

The goal of the evaluation and reporting system is to provide mean- 
ingful and comparable information about Title I projects at the school 
building, school district, state, and federal levels. The data are re-* 
ported from each Title I project. These data are entered on coding sheets 
and show identifying information for the school district, the building, 
and each student within the building. Information reported includes codes 
for the name, edition, form, level, and subtest of the tests used, the 
date tested, the number of teachers in the project, and individual scores 
on the tests. 



Participation statistics are aggregated (combined) for all students 
in Title I programs, but achievement outcomes are compiled only for those 
students with valid pretest and posttest scores. Project impact is mea- 
sured by performance of participants on achievement tests. Typically, 
students are tested in the fall (pretest) and again in the spring (post- 
test) after being in Title I classes for six or seven months. 

The various standardized tests available are broken down by form, 
level, and subtest. Forms are differing versions of similarly normed 
tests, level indicates level of achievement for which the particular ver- 
sion is recommended, and subtest focusses on a particular content area. 
Under most circumstances the same form and level of a given test should be 
administered at pretest and posttest times. 

As part of the Federal Title I reporting requirements. Title I a- 
chievement scores must be reported as Normal Curve Equivalents (NCEs). 
The rest score arrived at most directly is the raw score, found by simply 
adding the number of items on the test anawered correctly. However, raw 
scores are seldom if ever reported, because they provide no information 
that can be used in comparing achievement of different children. Instead, 
standardized tests provide norms tables which enable the user to convert 
the raw score to a percentile ranking, a grade equivalent or a standard 
score. Since one of the goals of Title I evaluation was to be able to 
compare outcomes across different states, a common metric which would 
allow such comparisons was developed by RMC Inc. (Tallmadge, 1976). This 
metric is known as the Normal Curve Equivalent or NCE. NCEs are most 
closely related to percentiles. Percentile ranking shows the percentage 
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of the norming sample scoring below a given raw score. NCE scores form a 
distribution which matches percentiles at the 1, 50, and 99 levels but, 
unlike percentile.^, distributes the intervals between scores evenly. For 
this reason NCE scores have been chosen as the metric for Title I evalua- 
tion. 

Evaluation measures the impact of the project. Impact is defined as 
a gain in the mean NCE score of the project from pretest to posttest. 
This is actually a measure of the difference between the students' NCE 
scores on the posttest and the scores the students would be expected to 
achieve without the benefit of Title I instruction. In effect, by conver- 
ting raw scores to NCEs the nationally normed sample becomes a "no-treat- 
ment control group" against which the Title I student's progress is mea- 
sured. For this reason, it is essential to good evaluation that pretest 
and posttest be given within the prescribed time limits in respect to the 
date the test was normed. 

Although the impact of Title I is measured at the project level, the 
following example shows how the NCE gain is computed for an individual 
student. A student selected for a Title I project in remedial reading is 
given a nationally normed reading test in the fall. His raw score is 62, 
which converts to the 22nd percentile, which converts to an NCE score of 
34. In the spring he takes the same test again and reaches percentile 30 
which converts to an NCE score of 39. NCE gain is the difference between 
39 and 34, or 5 NCE's. 
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The Iowa Title I Computerized Evaluation System 



Thousands of children participate annually in the Title I programs in 
the state of Iowa. To meet the goals of the Title I System without com- 
puter use has meant a time consuming manual task. Each record would have 
to be manually edited to convert the raw scores to NCE's. Additional test 
information regarding norm dates, appropriate use of out-of-level testing, 
and the like, would have to be appended to the records Tedious manual 
checks of each data field would have to be made; a :i an equally tedious 
compilation of statistics would need to be made. i.ie greatest disadvan- 
tage in this manual effort would be in the area of quality control. At 
any point in the process, human error is very likely to occur. Several 
papers on quality of Title I data identify the extent and seriousness of 
these errors (See, for example, Bozler, 1978; Crane and Maye, 1980; Elraan, 
1981; and Finley, 1981). Further, the volume and level of reporting would 
make the manual preparation of the feedback reports an enormous task. By 
contrast, the computerized system quickly and easily screens out most 
reporting errors and omissions, and easily compiles and prints the statis- 
tics for the various feedback reports. 

The Iowa Title I Evaluation System begins at the project/school dist- 
rict level with the administration of tests by teachers to Title I stu- 
dents. These tests are usually administered in the fall and again in the 
spring. Student raw scores on the tests are entered on coding sheets 
along with other pertinent data including an identifying number for the 
school district, the building, and each student in the project. The data 
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include codes for the name, form, level, and subtest of each test used, 
the dates tested, the number of teachers in the project and individual 
scores on the tests^ Figure 1 is a copy of the coding sheet used in Iowa 
to collect achievement information. These student coding sheets, accom- 
panied by a summary building-level coding sheet (see Figure 2) are then 
forwarded on to the state Title I agency. 

Figure 3 graphically shows how information flows through the Iowa 
Title I Evaluation System, that is, what happens to the coding sheets once 
they reach the state department of education. When the coding sheets are 
received by the state agency, they are manually screened for any errors or 
omissions. The records are then keypunched onto cards and transferred to 
tape. The resultant tape file of student data is then run through the 
MERGE program which merges the forms information into single student 
records. 



Insert Figures 1 through 3 about here 



An error report is created by this program to aid in the correction 
of records with errors. These records must be manually corrected (i.e. 
there is no computer program to correct them). The corrected records will 
then be resubmitted to the MERGE program. This cycle can be continued 
over and over until all errors have been corrected. 

Processing of data continues in the RAWCON program. The merged data 
set is run through this program and additional testing information is 
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FORM C (Rrad-RAW), Side 1 
Title I Evaluation Information for One Grade 



(3-10) 
(11-17) 
(18) 
(19-22) 
(23-2A) 
(25-27) 
(28-30) 
(31-33) 



(3A-41) 
(A2-49) 
(50-57) 
(58-65) 
(66-73) 

(18-21) 



(22) 
(23-26) 

(27^32) 

(33-^6) 



(37) 
(38-41) 
(42-A7) 
(48-A9) 

(50-51) 

(52-53) 
(54-57) 



I I I 



I I I 



I T 



1. a 

2 r~ 



10. 



11. 

12. 



3. □ 

A. (zr 



5. 

6. I 1 I 



7. dU 

8. r~rT 



Local Education, Agency Code 

Project Code 

Number of Aides 

Building Number 

Grade 

Group Code 

Minutes of Instruction Time Per Week 
Total Number of Days 



9. Student (S tu .)/ Ins true tor (Inst.) Information 



Stu. P Inst. 



# Stu. Inst. 



Group 1 
Group 2 
Group 3 
Group 4 
Group 5 



Croup 6 
Group 7 
Group 8 
Group 9 
Group 10 



Pretest Information 

Test Name and Year 

Form 

Level 

Subtest or Total Score 



Pretest Score Type Code 
(Raw Score - 1) 
Pretest Norm Date 



Month Day 



13. I I I i j I ] Pretest Administration Date 



Month Day Year 

14. Posttest Infonaation 

Test Name and Year 

Form 

Level 

Subtest 



15. m 



Month Day 



Posttest Score Type Code 
(Raw Score « l) 

Postteat's Norm Date 



17 . n~T 



Month Dav Year 



Title I Posttest Administration Date 



18. r~n 



15. cza 



Total Number of Title I Students it: 
this Grade and vho are Listed on Side 
2 of this Form that Received any 
Title I Instruction. 

How many of chese Students (Listed on 
Side 2) have both a Pretest and 
Posttest Score? 



2 0 . Student S elect ion Test Info rrzatlon For Th is FORM C 
Name of Test(s) and Date(s) of Administration 
Test 1 



Kanie&Yr. 



Form Level Subtest Mon. Bay Yr. 



Figure 1 : Sample 
Side 1. 
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vere selected for Title I participation usinj Test 

1? rest 2? 
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0 

(14-19) 

(20-25) 
(26-31) 

(32-37) 
(38-43) 
(44-49) 
(50-55) 
(56-61) 
(62-67} 
(6S-73) 

(14-19) 

(20-25) 

(26-31) 
(32-37) 

(38-43) 
(44-49) 
(50-55) 
(56-61) 
(62-67) 
(68-73) 

(14-19) 
(20-25) 
(26-31) 
(32-37) 
(38-43) 
(44-49) 



PUPIL 



5. 



7. 



8. 



10. 



11. 



12. 



13. 



14. 



15. 



15. 



17. 



18. 



19. 



20. 



21. 



22. 



23. 



24. 



2b. 



26. 



27. 



28. 



PRETEST 



1. 
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2. 


1 


1 


3. 






1 


4. 
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(50-55) 
(56-61) 
(62-67) 

(68-73) 
(74-49) 

© 

(14-19) 

(20-25) 

(26-31) 

(32-37) 

(38-43) 

(44-49) 

(50-55) 

(56-61) 

(62-67) 

(6$-73) 

(74^) 

(14-19) 

(70-2S) 

(26-31) 

(32-37) 

(38-43) 

(44-49) 

(50-55) 

(56-61) 

(62-67) 

(68-73) 

(74-79) 
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29. 



30. 



31. 
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34. 



35. 



36. 



37, 
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39. 
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41. 



42. 
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44. 



45. 



46. 



47. 



48. 



49. 



50, 
51. 



52. 



53. 



54. 



55. 



PRETEST 
SCORE 



POSTTEST 
SCORE 
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11 
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FORM B (Read) 
LEA and Reading Project Summnry Sheet 



Direct 1 ons : Refer to pages 29 - 30 in the boo"klct Instructions for 
Subinitting Annual Evaliiation Report FY 1981 . 



1. 



DPI Local Education (LEA) Code 



2. 11,1 I Reading Project Code 



3. 



1 j Subject Code (Reading = 1) 



5. 



Reading Project (K-12) Cost from Title 1 
Funds Only (to nearest whole $). 



Total Reading Project (K-12) Cost from 1 
Funds and LEA Supplemental Funds (to ncc 
■whole $). 



7. 



Total LEA Enrollment (K-12) 



Total Title I Reading Project Enrollment (K-12) 



Figure 2: Sample Form B. 
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Figure 3: Iowa Title I system. 
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THE REPORT 
PROGRAM 



BUILDING 

SUMMARY 




added to the record. This information comes primarily from a computer 
table called the Test Characteristics File which lists important informa- 
tion about the various tests the teachers have administered to their stu- 
dents. -Examples of information added are the nojftiinal norm date of the 
test, whether norms for the test are empirical, and the actual norm date 
for the test — all vital pieces of information needed to further evaluate 
the student test data. If the program cannot attach this information to 
the record (which happens if the test in question is not in the system, or 
if the teacher-supplied information is ambiguous so that the appropriate 
Test Characteristics record cannot be accessed), error flags are set in 
the record and further processing of the test data will be limited. RAW- 
CON also converts the reported student test scores to NCE scores. An 
error/statistical report is produced by this program to detail any errors 
found, and to aid in the documentation of the processing. 

The updated merged record file is then run through lARPT, the repn-t 
program. The function of the lARPT program is to produce the various 
reports needed for Title I evaluation. These reports* include building- 
level summary (for school and teacher feedback), district-level summary 
reports (for district-level feedback), and district-level error summa- 
ries. Examples of these reports are in Figures 4 to 6. 



Insert Figures 4 through 6 about here 



In addition to these programs, the Iowa System includes two other 
programs used to update (NTABUPD) and print (NTABPRT) the Norms Tables 
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STATE OF IOWA 

SUniiARY OF TITLE I PROPPAM IMPACT PAGE \ 

BUILDING LEVEL REPORT— 1 960-61 SCHOOL YEAR 
DISTRICT! ^OAI BUILDING CODEt 6106 

SUBJECT MATTER AREAt READING 









EVA 


L U A T I 


0 N 


RESULT 


S 












NUMBER 


PERCENT 


PRETEST 


POSTTEST 


GAIN 


GRADE 
2 


TOTAL 
PARTIC- 
IPANTS 
3 


NUMBER 
PRE C 
POST 
3 


WITH 
VALID 
SCORES 
3 


WITH 
VALID 

SCORES 

toox 


MEAN 

NCE 
54.5 


EQUIV- 
ALENT 
XXLE 
59'A 


MEAN 
NCE 
60. 1 


EQUIV- 
ALENT 

6ex 


MEAN 
NCE 
GAIN 
♦ 5.6 


3 


3 


3 


3 


100/; 


36.3 


Z9y. 


46.1 


43>: 


♦ 7.6 


A 


4 




4 


100/; 


46.0 


42/: 


52.8 


5sy. 


♦ 6.6 


5 


A 


4 


4 


100); 


27.9 


15/: 


36.2 


Z9'A 


♦10.3 


6 


3 


3 


3 


100/: 


29.7 




45.9 


42/: 


♦16.2 



***<***5UMMARYli<<<<Ki« 

* OVERALL BUILDING EVALUATION RESULTS K 
^ TOTAL NUMBER OF PRETEST POSTTEST GAIN « 
^ NUMBER OF STUDENTS WITH EQUIV- ' EQUIV- "^EaTT' ^ 
H PARTICIPANTS VALID TEST PERCENT MEAN ALENT MEAN ALENT BUILDING ^ 
^ (GR. 2 - 12) SCORES ANALYZED NCE /HE _ [iC L. ;:iLE GAIN « 

♦* <7 17 100. OX 39.0 30X 46.2 47X ♦ 9.2 « 

* K 



FIguro 4 : Building Report Page 1 
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STATE OF IOWA 

SUMMARY OF TITLE I PROGRAM TMPA(; T PAGE 2 

DISTRICT LEVEL REPORT— 1 980-61 SCHOOL YEAR 
DISTRICTi 
SUBJECT MATTER AREAt READING 









P V A 


L U A T I 


ON R 


E S U L T S 








TOTAL 
PARTIC- 
IPANTS 




NUMBER 


PERCENT 


PRETEST 


POSTTEST 


GAIN 


GRAD^ 


NUMBER 
FRE C 
POST 


WITH 
VALID 
SCORES 


WITH 
VALID 
SCORES 


MEAN 


EQUIV- 
ALENT 

21X 


MEAN 
AS. 7 


EQUIV- 
ALENT 

42X 


MEAN 
NCE 
GAIN 
♦12.8 


2 


13 


13 


13 


100X 


32.9 


3 


18 


17 


17 




35.8 


25X 


43.1 


37X 


♦ 7.3 




19 


18 


18 


95X 


3A.0 


22X 


42.4 


36/^ 


♦ 8.4 


5 


3^ 


31 


31 


91X 


^1.2 


3<^X 


48.6 


47/i 


♦ 7.4 


6 


31 


Z9 


29 


9^X 


10.1 


3X 


49.2 


49^ 


♦39.1 



'««««SUM MARY«««KK 

' OVERALL DISTRICT EVALUATION RESULTS x 

f TOTAL NUMBER OF PRETEST POSTTEST GAIN « 

NUMBER OF STUDENTS WITH EQUIV- EQUIV- MEApT" « 

PARTICIPANTS VALID TEST PERCENT MEAN ALENT MEAN ALENT DISTRICT « 

g - <g) SCORES ANALYZED tiC£. XILE NCE XI LE GAIN « 

M 

115 108 93. 9X 29.8 17X 46.5 ^YA ^16. 7 « 



Figure 5; District Report - Page 2. 
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SUMMARY OF EVALUATION PROCEDURES PAGE 3 

DISTRICT LEVEL REPORT— 1 9ft0-fi1 SCHOOL YEAR 

DISTRICTi ^0^3 TOTAL NUMBER OF PARTICIPANTS 115 

(GRADES 2-1 2 ONLY) 

SUDJECT MATTER AREAi READING 

» NUMBER « 44 

« OF STUDENTS « EVALUATION PROCEDURE N 

^ AFFECTED « ^ 



« « STUDENTS HAD NCES OR SCORES FROM A CONVERTIBLE TESTi « 

^ 115 (100X) « YES M 

MM If 

^ « TEST BATTERY USED AT PRETEST AND POSTTEST TIMES* K 

» 115 (100X) « SAME BATTERY M 

* « K 
« « SUBTEST USED AS PRETEST AND POSTTEST i m 
« 115 (tOOX) « SAME SUBTEST M 

» « MISSING OR INCONVERTIBLE STUDENT PRETEST OR POSTTEST SCORES « « 

* 108 ( 9^X) » NO N 
« 7 ( 6X) « YES 
« « STUDENTS SELECTED USING PRETEST i M 
M 1 15 ( 100X) K NO M 

« NORM DATES AND TEST ADMIN. DATES FOR PRETEST AND POSTTESTi « 

« 115 (100X) « ALL INFORMATION AVAILABLE K 

« « PRETEST, POSTTEST GIVEN AT PROPER TIME RELATIVE TO NORM DATESiM 

« 115 (100X) « YES « 

*i « ^ 

MM N 

N 

*< SUMMARY OF EVALUATION PROCEDURES i x 

103 ( 9^X) STUDENTS HAD DATA ACCEPTABLE FOR ANALYSIS « 

^ 7 ( 6X) STUDENTS HAD DATA HOT ACCEPTABLE FOR ANALYSIS » 



Figure 6: District Error Summary Page 3. 
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file. 

The computer programs in the Iowa Title I Evaluation System are writ- 
ten in COBOL and designed to run on an IBM-370 (or compatible) system. 
Files are stored on standard labeled computer tapes. 

In recent months work has been completed on the documentation of the 
system^ A Technical Manual and User's Guide are available to assist users 
to operate , maintain, and enhance the system. 

Some Data on System Impact 

And what of Iowa's experiences with the system. Are they pleased 
with it? What has been the payoff for them? While we do not pretend to 
be able to speak for the Iowa Title I staff, there are some indices which 
we have collected which speak for themselves. Where score conversion and 
aggregation accuracy is concerned, the type of errors previously noted 
have been totally eliminated. Errors that have been discovered in the 
system have been attributed primarily to (a) errors or omissions in the 
norms tables; (b) inaccurate coding of test, form, level, and subtest; and 
(c) data entry errors. As errors have been discovered in the norms 
tables, they have been corrected. For the most part, errors of this type 
have been relatively minor considering the 15,000 tables available for 
.access. Inaccuracy in test information coding will be detected automatic 
cally if the raw score is greater than the maximum score possible on the 
miscoded test. Visual scanning of the output may also detect "suspicious" 
looking scores that would lead one to question accuracy. 
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However, if the raw score can be converted and the output looks "rea- 
sonable," an error caused by inaccurate coding will go through the system 
undetected- The error rate from this source is unknown, and much depends 
on the pre-edit and post checking steps that are taken. Data entry errors 
are treated the same way as errors due to inaccurate coding. It is as- 
sumed that some error (unknown) will go through due to undetected errors 
ift data entry. The amount of error will depend on the accuracy of entry 
verification steps that are taken. 

J 

Where validity is concerned, tables 1 through 7 summarize the results 
across years. In 1978, 27,905 student reading scores were submitted for 
processing (Table 1). Of those, 25,670 (92.0%) had used tests that were 
included in the system; 20,386 (73. IZ) used tests in the system and had 
both a pretest and a posttest score; and 16,909 (60.6Z) met all the cri- 
teria for inclusion in the analysis."^ 



Insert Tables 1 through 7 about here 



In Table 2 looking at only the 20,386 students having both a pretest 
and posttest score that could be processed by RAWCON, the number and per- 
cent meeting each criterion for inclusion is given: (1) 99, 7Z had taken 
the s^me pretest as posttest; (2) 89.7% had been tested within 1 month of 



Criteria included (a) used same pretest and posttest battery, (b) tested 
within one month of empirical norm date, and (c) selection was not based 
on the pretest. 

13 



15 



Table 1 

General Summary of How Iowa Students Were Selected 
for Analysis Based on Total Number of Students Whose 
Scores Were Submitted for Analysis 

1978 



Reading Mathematics 
Selection Categories Number Percent Number Percent 



Total students whose scores we;re 

submitted for processing 27,905 100.0 5,203 100.0 

Students having scores 

that could be processed^ 25,670 92.0 4,361 83.8 

Students having both a pretest 

and a posttest score 20,386 73.1 3,294 63.3 

Students meeting criteria 

for inclusion in analysis^ 16,909 60.6 2,421 46.5 



^ Only those student scores from among an estimated 95Z of the most fre- 
quently used tests in Iowa could be processed. Score conversion tables 
were obtained for these tests. Raw scores reported for the remaining 5Z 
could not be processed without the appropriate score conversion tables. 

^ Criteria included (a) tested with same test pre and post, (b) tested 
within one month of empirical norm date, and (c) selection not based on 
pretest. 
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Table 2 



Summary of Number of Iowa Students Meeting Individual 
Criteria for Inclusion in Analysis Based on Number of 
Students Having Both a Pretest and a Posttest Score 

1978 



Reading Mathematics 
Selection Categories Number Percent Number Percent 



Students having both a pretest 
and a posttest score 


20,386 


100.0 


3,294 


• 

100.0 


Students having the same 
pretest and posttest 


20,320 


99.7 


3,286 


99.8 


Students tested within one month 
of empirical norm date 


18,282 


89.7 


2,706 


82.1 


Students not selected with pretest 


18,488 


90.7 


2,966 


90.0 


Students meeting criteria 
for inclusion in analysis^ 


16,909 


82.9 


2,421 


73.5 



^ Criteria include (a) tested with same test pre and post, (b) tested 
within one month of norm date, (c) selection not based on pretest. 
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Table 3 



General Summary of How Iowa Students Were Selected 
for Analysis Based on Total Number of Students Whose 
Scores Were Submitted for Analysis 



1979 



e . ^ ^ Reading Mathematics 

Selection Categories Number Percent Number Percent 



100.0 



Total students whose scores were 

submitted for processing 34,323 100.0 7,642 

Students having scores that could 

be processed 32,928 95.9 6,863 89.8 

Students having both a pretest 

and a posttest score and ' 19,607 57.1 3,230 42 3 
available norms tables^ 

Students meeting criteria 

for inclusion in analysis^ 17,976 52.4 2,983 39.0 



b 



The reduction over 1978 was due to the addition of norms tables for new 
editions of tests being only partially completed by the time the SEA 
report had to be run. 

Criteria included (a) tested with same test pre and post, (b) tested 
within one month of empirical norm date, and (c) selection not based on 
pretest . 



2d 



18 



Table A 



Summary of Number of Iowa Students Meeting Individual 
Criteria for Inclusion in Analysis Based on Number of 
Students Having Both a Pretest and a Posttest Score 

1979 



Reading Mathematics 
Selection Categories Number Percent Number Percent 



Sfiidpnti*; having Kn^Vi a nretest 
and a posttest score 


19,607 


100.0 


3,230 


100 


.0 


Students having the same 
pretest and posttest 


19,594 


99.9 


3,198 


99 


.0 


Students tested within one month 
of empirical norm date 


18,872 


96.3 


3,109 


96 


.3 


Students not selected with pretest 


18,634 


95.0 


3,065 


94 


.9 


Students meeting criteria 
for inclusion in analysis^ 


17,976 


91.7 


2,983 


92 


.4 


^ Criteria include (a) tested with 
within one month of norm date, (c 


same test pre and post 
) selection not based 


, (b) tested 
on pretest. 
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Table 5 



General Summary of How Iowa Students Were Selected 
for Analysis Bag.ed on Total Number of Students Whose 
Scores Were Submitted for Analysis 



1980 



Selection Categories 


Reading 
Number Percent 


Mathematics 
Number Percent 


Total students whose sc,ores were 
submitted for processing 


33,A27 


100.0 


7,906 


100.0 


Students having scores that could 
be processed 


33,197 


99.3 


7,466 


94.4 


Students having both a pretest 
and a posttest score 


. 26,461 


79.2 


5,575 


70.5 


Students meeting criteria 
for inclusion in analysis^ 


25,483 


76.2 


5,277 


66.7 



^ Criteria include: (1) have scores that could be processed, (b) have pre- 
and posttest scores, (3) tested with same test pre and post, (4) tested 
within one month of empirical norm date^ and (5) selection not based on 
pretest . 
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Table 6 



Summary of Number of Iowa Students Meeting Individual 
Criteria for Inclusion in Analysis Based on Number of 
Students Having Both a Pretest and a Posttest Score 

1980 



Reading Mathematics 
Selection Categories Number Percent • Number* Percent 



Total students having both a • 

pretest and a posttest score 26,461 100.0 5,575 100.0 

Students having the same 

pretest and posttest 26,461 100. o' 5,571 99.9 

Students tested within one month 

of empirical norm date 25,683 97.1 5,298 95.0 

Students not selected with pretest 26,196 99.0 5,537 99.3 

Students meeting criteria 

for inclusion in analysis^ 25,483 96.3 5,277 94.7 



^ Criteria include: (a) tested with same test pre and post, (b) tested 
within one month of empirical norm date, and (c) selection not based on 
pretest . 
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Table 7 C 



ITTlfTlsa "T^V 


of Number 


and Percent 


of Students 


Meeting 


Validity 


Criteria 


in Iowa Over 


a Three Yea 


r Period^ 


Year 


Reading 


Mathematics 


Ending 


Number 


Percent 


Number 


Percent 


1978 


16,909 


82.9 


2,421 


73.5 


1979 


17,976 


91.7 


2,983 


92.4 


1980 . 


25,483 


96.3 


5,277 


94.7 



^ Based on students who took a test included in the gysten 
(99X in 1980) and students having both pretest and post- 
test scores (79X in 1980). 
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the empirical norm date; and (3) 90.7% had not been selected on the pre- 
test. The percent of students meeting all three criteria was 82. 9Z. This 
can be thought of as an index reflecting the extent to which the models 
have been correctly implemented. 

* Similar information is given for 1979 in Tables 3 and 4 and for 1980 
in Tables 5 and 6. The overall picture is presented in Table 7 and shows 
progressive increases for reading from 82.9% in 1978 to 91.7% in 1979 and 
to 96.3% in 1980. Similarly, the index for mathematics goes from 73.5% in 
1979 to 92.4% in 1978 to 94.7% in 1980. 

Other interesting facts include the following: 



Reading (over the three-year period) 

1. The percentage of students tested within one month of the 
empirical norm date went from 89.7% to 97»1%. 

2. The percentage of students not selected on the pretest rose 
from 90.7% to 99.0%. 

i 

3. The' percentage of students having scores that could be pro- 
cessed (i.e., a test in the RAWCON system was used) went 
from 92.0% to 99.1%. 

Mathematics (over the three-year period) 

1. The percentage of students tested within one month of the 
empirical norm date went from 82.1% to 95.0%. 

2. The percentage of students not selected on the pretest rose 
from 90.0% to 99.3%. 

/ • 

3. Tne percentage of^ students having scores that could be pro- 
cessed (i.e., a test in the RAWCON system was used) rose 
from 83.8% to 94.4%. 

Development of the RAWCON system has not been without its problems. 

It^ depends heavily upon the proper coding of test identification (including 
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level, form, and subtest) and upon careful hand pre-editing of the data as 
well as checking of output. However, Iowa's experience attests to gain^ 
that can be made in assuring the quality of the vast majority of student 
data collected at the SEA level. 
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